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Abstract This paper deals with the parameter estimation problem of the Single- Input-Single- 
Output (SISO) switched Hammerstein system. Suppose that the switching law is arbitrary but 
can be observed online. All subsystems are parameterized and the Recursive Least Squares (RLS) 
algorithm is applied to estimate their parameters. To overcome the difficulty caused by coupling of 
data from different subsystems, the concept intrinsic switch is introduced. Two cases are considered: 
i) The input is taken to be a sequence of independent identically distributed (i.i.d.) random variables 
when identification is the only purpose; ii) A diminishingly excited signal is superimposed on the 
control when the adaptive control law is given. The strong consistency of the estimates in both 
cases is established and a simulation example is given to verify the theoretical analysis. 

Key words SISO switched Hammerstein system, RLS algorithm, intrinsic switch, diminishing 
excitation, strong consistency. 

1 Introduction 

Because of importance in engineering applications, the identification and control of switched 
systems have been active research areas for years ^ . Concerning parameter identification of switched 
systems, a survey is given in [5]. 

The switched systems can roughly be divided into two classes: systems with an arbitrary switch- 
ing mechanism and systems governed by a constrained switching law, such as the Markovian switch- 
ing rule. In the existing literature there are many papers on Markov Jump Systems, see, e.g., |3] 
and the references therein. 

By using the algebraic geometry as the key tool and under the assumption that the number of 
subsystems, the subsystem orders, and the switching sequence are unknown, the author of 4 pro- 
vides an algorithm to recursively estimate the unknown parameters of the discrete-time Switched 
Auto-Regressive eXogenous (SARX) model, and gives the algorithm a convergence analysis. How- 
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ever, in the convergence analysis given in ^ no unpredictable disturbance is taken into account, 
despite the examples given there are with noises. While the authors of |5] tackle the SARX model 
with noises; they suggest an algorithm that alternates between data designation to submodels and 
parameter update, but do not prove its convergence. 

In this work, we consider parameter estimation of the Single-Input-Single-Output (SISO) switched 
Hammerstein system and assume that the switching law is arbitrary but can be observed online. 
We will handle two cases: 

i) In the case where identifying the system is the only concern, we take the system input as a 
sequence of i.i.d. random variables. It is assumed that the nonlinear function of each subsystem 
can be expanded to a linear combination of continuous base functions. 

ii) In the case where the adaptive control has been designed for the system, we apply the 
diminishing excitation technique!^ to recursively estimate the unknown parameters. In this case, 
we assume that the continuous base functions, a linear combination of which the nonlinear part of 
each subsystem can be expanded to, are monomials. 

The rest of the paper is organized as follows. The problem is formulated in Section 2, and the 
parameter estimation algorithm is constructed in Section 3. In Section 4 we prove that the estimates 
given by the proposed algorithm are strongly consistent, and then we provide a simulation example 
in Section 5. Some concluding remarks are given in Section 6. Appendix at the end is used to load 
proof details. 



2 Problem Formulation 

The SISO switched Hammerstein system considered in the paper is presented in Fig. 1. It 
contains a finite number of Hammerstein subsystems, each of which consists of a static nonlinear 
G (•) followed by an ARX subsystem in cascade. 
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Fig. 1. SISO Switched Hammerstein System 



We assume that there are J subsystems, and consider the case where the switch mechanism is 
available. To be precise, the mapping A (•) 

n4{1,2,... ,J} 

k Afc 

can be observed online, where N represents the set of all nonnegative integers, and Afc denotes the 
serial number of the Hammerstein subsystem that operates at time k. Besides, the orders p, q of all 
ARX subsystems are supposed to be the same and known. Moreover, Gj (•), Vj G {1, • • • , J}, can 
be expressed as a linear combination of r basis functions: 5i (•) j ' ' ' i5r (■)■ 
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By setting 



Ax, {z) =l + a<^'='z + --- + 



A ■ 

2 

1=1 

the system can be described as 

' Vk = Gx, (ufe) , 



^Xk (z) Vk+i = Bx, [z) Vk + Cfe+i, fc > 0; 
^ Mfc = 0, Wfe = 0, ^fc+i = 0, Uk+i = 0, /c < 0, 



(1) 



where Uk is the input, vu is the unmeasurable internal signal generated by Gx, {■), Vk is the output, 
^fc is the driven noise, and z denotes the backward shift operator, zyk = Uk-i- 
On the other hand, we set 



A 



hxh 



7 (^fc + h-l) (^fc + fl-l) 



hxr 



= [1 • • • 0]ix/i' and ii^ = [51 (uk) ■ ■ ■ gr (wfc)]ixr' "^^^ere h = max{p,q}, aj"' = 0, b!^ = 0, for 
I > p,m > q,j £{!,■■ ■ , J}. Then System ([T]) can be expressed in the state space form as follows: 



yk = G^'xk, 

4-[yo ••• 0]i,^ = [0 ••• 0],,^. 



(2) 



Remark 1 It is seen that A**"' and i?**"' tofce values in the finite sets, which will be denoted by 
{A^'\--- ,A<^i'} and {B<^',--- ,B<^='}, respectively. 
We make the following assumption on the system. 

(HO) For each j G {1, 2, • • • , J}, A""'^ ({j}) is an infinite subsequence of N, and 



A-M{ji})nA-i({j2}) 



V 1 < il 7^ J2 < J, 



Remark 2 5?/ (HO) we preclude those subsystems that only operate for a finite number of times; 
this is reasonable when processing parameter identification task. 
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For System ([T|), the parameter estimation problem is to recursively estimate the unknown pa- 
rameters ai\--- ,a^/\ bi\--- ,6^^', Ci\--- ,c<?', Vj G {1, • ' ' ^J}: based on the designed input 
{ufcj^Q and the measured output {yk}'kLi- 



> oo. It is worth noting that y,u),, is generated 

• • ,p — 1}, is not necessarily the output of the jth 



3 Estimation Algorithm 

Let j e {1,2, • • • , J} be arbitrarily fixed. By (HO) we are able to write ({j}) = {^t ^}T=o 
with fcj^' < whenever Q < I < s. Clearly {^t^'}^Q denotes all the times at which the jth 
Hammerstein subsystem operates; we have k[^^ 
by the jth subsystem, while y.u) ,, Vd G {0, 
subsystem. 

Let us introduce a concept named intrinsic switch. Corresponding to y,u) ■ ■ ■ yi.{j) 
set = [riQ • • • , where ri^^"'', d € {0, • • • ,p — 1} denotes the serial number of the 

Hammerstein subsystem that generates 2/^,(3) It is seen that n'^x*' is among Jp = K different 
combinations. From now on, we say an intrinsic switch occurs whenever n*^"*' changes. Evidently, 
we may partition {ijt^n ^^^'^ ^ subsequences {t\^\ m > 0}, k = 1, • • • , if , such that for each k £ 



+1- 



we 



{I,--- ,^}, 



is independent of m. It is noticed that there exists at least one k £ {!, - ■ ■ , K} 



such that {tl^\m > 0} is an infinite subsequence of {ijt^o- 

Remark 3 The term "intrinsic switch" should be distinguished from "switch"; "switch" indi- 
cates the behavior that System (jlj jumps from one Hammerstein subsystem to another. 

By the notations introduced above, we know the jth Hammerstein subsystem works by the 
following equation: 



{b'f 



1=1 



(3) 



Denoting by 



and 



M) A 



M ■■■ 

the parameters in the jth regression subsystem and the regressor, respectively, we rewrite ([3| as 

2/,a)+i-^"'>^''+4<^)+i, t>0. (4) 
Let {9l'^}t>i be the estimates of 6*"^ Set 6l^'> arbitrarily and = a'^'/ with some £ (O, i). 
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The RLS algorithm ^ estimating 6''^' is defined as follows 



p. 



p(j) p(j) p(j) 



<y5 



By (§ and it follows that {Pi^iY^ = ELo^^Vf" + 



(5) 
(6) 
(7) 

(8) 



The estimates of h''{\ 



conditionsl^iaii. 



() 

c^i \ • • • , can be derived from {^J^'}t>i under some identifiable 



4 Convergence Analysis 

Let (ri,^,P) be the basic probability space. The following assumptions are to be used. 
(HI) {^k,-^k} is a martingale difference sequence^^ with 



< oo a.s., /? > 2, 



where {^k} is a sequence of nondecreasing sub cr-algebras of 
(HI') {£,k,'^k} is a martingale difference sequence with 

suPfc ICfcl < < oo a.s., 

where is a positive constant, and {^k} is a sequence of nondecreasing sub ti-algebras of 
(H2) limfe_i.oo j: Ei=i = P{ > a.s., where is a constant. 

(H3) {l,5i (•),••• ,(?r-(')} is linearly independent over some interval [a,b], and gi{-), G 
{1, • • • , r} , is continuous on [a, 6]. 

t 

(H5) There exists a 7 > such that as t -> cxd, ^ y^j^, = O (f) a.s., Vd G {0, • • • , p — 1}. 

i=0 

(H5') There exists a finite positive integer n such that . . . ^(w) |j < 1^ VA"™> G 

{A*^', • • • , A^'^i'} , for m = 1, • • • jfi, where ||-|| is the induced 1-norm: 

II All ^ maxi<d,<,, Y!1^^ l^'ii'i^l' = («<iidj,,x£. e M^i^^^ 

Remark 4 A'^oie </ia< (H5'), as well as (H5), is a condition concerning stability of System ([T|). 
Stability oj time-varying systems is discussed in Ulf by introducing an assumption similar to (H5'). 
For convenience of citation, we list a lemma here: 

Lemma 1 (Theorem 2.8 oJ 161) Let {Xk,'^k} be a matrix martingale difference sequence and 
let {Mkj'^k} be an adapted sequence of random matrices with ||Mfe|| < 00 a.s., Vfc > 0. // 



sup,S[||X,+ir|^fe] <oo a.s. 
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for some a G (0,2], then as k ^ oo 
k 

Y,Ma^+l = o(^Sk{a)logi+^{s^{a) + e)) a.s., V77 > 0, (9) 



i=0 



where Sk (a) = (l]i=o P'^^l 

We give the convergence analysis of Algorithm ([5])-(|8]) for two cases as follows. 

4.1 Case I — Using the i.i.d.-Type Input 

The i.i.d.-type input is taken satisfying: 

(H4) {uk} is a sequence of i.i.d. random variables with density p{-), which is positive and 
continuous over [a, 6], and vanishes outside [a, 6]. Besides, {uk} is independent of {£,k}- 
Before proving our first result (Theorem 1), we need lemmas 2-5. 
Lemma 2 (Lemma 1 of 111) If (H3) and (H4) hold, then 

R = E[gi {uk) - Ail • • • gr {uk) - i^rY[gi {uk) - /ii • ■ ■ ffr {uk) ~ Mr] > 0, 

where = Egi (uk), V/ G {1, • • • ,r}. 

Lemma 3 // (HI'), (H3), (H4), and (H5') hold, then yk = O (!) a.s., as k ^ 00. 

Proof The proof is straightforward since System ([2]), and thereby System Q, is a contraction 
mapping. □ 

By AJ^'ax(^) s-'^d Aj^ijj(i) we denote the largest and smallest eigenvalue of (Pt+i) , respectively. 
The following two lemmas are motivated by Theorems 4.1 and 6.2 in respectively. 

Lemma 4 Assume that (HO) and (HI) hold, and that u„ is ^n-measurahle for all n > 0. 
Then as t 00 the convergence (or divergence) rate of the estimate given by Algorithm ([5])-(|8| is 
expressed by 



\ogX[iUt)(loglogMAUt)f^-'^^ 



^ ^ ^ f 0, x^O; 
where 5 (x) = < with arbitrary constant c > 1. 

I c, a; = 0, 

Proof Applying the same method as that used in the proof of Theorem 4.1 in [B], we arrive at 
the desired result. □ 
Lemma 5 // (H0)-(H4-) hold, then the following assertions are true. 

1) It holds that 

\0) (f\ 

liminf ""'"^ ' > a.s. (11) 

t^oo t 

2) If, in addition, (H5) holds, then the RLS estimate given by Algorithm (|5])-([8]) is strongly 
consistent and has the following convergence rate: 



.,.-9>^-f.of'°""°"°"'""') (12) 



3(3) 
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Proof Analogously to the proof of Theorem 3 in [7], which is motivated by the proof of Theorem 
6.2 in [S], we give the detailed proof of the lemma in Appendix. □ 
We are now in a position to give and prove our first theorem. 

Theorem 1 // (HO), (HI'), (H2)-(H4), and (H5') hold, then the RLS estimate given by 
Algorithm ([5])-([8| is strongly consistent and has the following convergence rate: 

||0^^7i-0«'f = 0((logt)A) a.s. (13) 



Proof Combining Lemmas 3 and 5 yields the theorem. 



□ 



4.2 Case II — Integrating the Given Adaptive Control with a Diminish- 
ingly Excited Signal 

Assume the following assumption holds: 
(H3') gi (x) = x\ Vx G K, V/ e {1, • • • , r}. 

Let {sk} be a sequence of i.i.d. random variables with continuous distribution, and let {e/t} be 
independent of {^k} with Esk — 0, Eel ~ ^' ^^"^ kfel — ^o, where (5o > is a constant. DefineE^l 



£fc 

J,e/2 



(14) 



with e > sufficiently small such that the interval (5,1 — {M + 1) re] is nonempty, where M 
Jp+q-1. 

Without loss of generality, we assume {^k} is rich enough such that ^k, w^''' G ^k- Set ^f;_i 
<y {61 , < zi < fc, , < Z2 < - 1}. 

Motivated by Theorem 6.2 in [Q, we introduce the following hypothesis. 



(H4') The given adaptive control v:^' is ^^. -^-mcasurable, i.e. 
O (1) a.s., as — )■ cxD. 

The diminishing excitation tcchniquel^ suggests to take 

A (c) I (d) 



' < G Vfc, and = 



(15) 



as the actual input, where v\, is given by (14 1. 
Define[l2] 



C/(fc) 



cWk' 



r-2 



and Vk — 



The following lemma is a corollary of Lemma 4 in [12]. 

Lemma 6 Let {ks}'^Q be an infinite subsequence of {fc}^g and let {tn}^^Q be an infinite 
subsequence o/{t}^Q. If (H4-') holds, then we have 



T^Y.^ (^^) - {vk, - EvkJU^ {ks) > Co/ a.s. 



(16) 



s=0 
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for all large enough tn, where cq > may depend on sample paths. 



Proof Noticing (H4'), we obtain (16 1 by investigating its counterpart in [T2] with s replaced 
by r and S set as 0. □ 

Modified from Theorem 2 in [T2], we have the following theorem in parallel to Theorem 1. 

Theorem 2 // (HO), (HI'), (H2), and (H3')-(H5') hold, then the RLS estimate given by 
Algorithm ([5])-([8]) is strongly consistent and has the following convergence rate: 



qU) 



3(3) 



o 



\ogt 



a.s. 



{M + l)re 



(17) 



Proof ( outline ) Bearing a resemblance to the proof of Lemma 5 (see Appendix) , for simplicity 
of notations, we omit the superscript (j). Reviewing the proofs of Lemma 5 and Theorem 1, we see 
that to prove the present theorem, it suffices to show 



lim inf — A 

t-»-oo t" 



[fj[ ) >0 a.s., Vae { l-,l-{M+l)re 



(18) 



where fi = Yll^i {z)(pi. Applying once again the method of reduction to absurdity and the 
procedure of subsequence partitioning and seeking (see Remark 6 at the end of Appendix) as that 
used in the proof of Lemma 5, and reasoning similarly to the proof of Theorem 2 in pL2^ with 
Lemmas 1 and 6 used repeatedly, we obtain the expected result. □ 



5 Simulation Example 

Consider the following system: 



2/2t = l-l2/2t-i - 0.28?/2t-2 + 0.5u2t^i - 1.5u: 



2t-l 



2(0.5u2t- 



1.5u 



2t-2 



2u2t^2)+^2t, 



2/2t-i = 0.8y2i-2 - 0.15?;2t-3 + 0.4u2(-2 



- 3(0.4u2t-3 + l-6u^t_3 - O.Sui_^) + C 



2t~l, 



J>2. 




(19) 



Let us verify (H5') for System (19) first. It is seen that 

I -0.28 or 

-0.15 J ' 

Using MATLAB to calculate, we find that (H5') holds with n = 9. 

We now assign the noise {^fc} and the excitation source {efe}, and set the initial values for 
Algorithm Let {^k} k>3 i.i.d. and uniformly distributed on [—3,3]. Take {£k}k>i to 



be i.i.d. and uniformly distributed on [—2,2] and independent of {^k}- Set 9q = 
' = Pq^' = 0.2/8, where Ig denotes the 8x8 identity matrix. 
Two types of input are taken separately to serve the parameter estimation task: 



and 



p(i 
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Case I Set Uk ~ £k- It is noticed that all the conditions (HO), (HI'), and (H2)-(H4) are 
fulfilled. Thus, by Theorem 1, the estimate given by Algorithm ([5])-(|8]) is strongly consistent. 

On the other hand, using the designed input and the collected output to execute Algorithm 
(|5|-([8]) twice, each running 2000 steps, we obtain the recursive estimation for the parameters of 
System (19 1 as shown by Fig. 2. 
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Fig. 2. Simulation results (I) 

Case II Disregarding the specific control cost, we suppose that uJT' = y'^+\y\ i|+i ^^^'^ given 
adaptive control at time k. Set = -j^0oT and Uk = uj,"' + wj.**' — yi+\^^ + fcrmn- Clearly, all 
the assumptions needed by Theorem 2 hold; hence, by Theorem 2, the estimate given by Algorithm 
(|5|-([8| is strongly consistent. 

In this case, the corresponding simulation results are presented in Fig. 3. 
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Recursive estimation of the parameters of the regression subsystem (2) 
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Fig. 3. Simulation results (H) 
It is seen that in either case the simulation outcome convincingly validates the theoretical 
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analysis. 

Remark 5 To derive the estimates o/ 62"^', c^^', C2^', Cg^', 6^^', 62^', c^^', Cj^', Cg^' from the sim- 
ulation results, we need to introduce appropriate identifiable conditions, see, e.g., or or JV^ 
for details. 

6 Concluding Remarks 

In this study, we apply the RLS algorithm to estimate the parameters of each parameterized 
subsystem of the SISO switched Hammerstein system, and under reasonable conditions we establish 
the strong consistency of the estimates. Especially, in the second case, by using the diminishing 
excitation technique, we also cater to adaptive control demands. For further work, it is of interest to 
consider the case where the switch mechanism is not exactly available and to weaken the restrictions 
on the noise, for example, to remove the boundedness assumption. It is also of interest to consider 
the closed- loop identification problems with control costs associated . 



7 Appendix 



Proof of Lemma 5 For simplicity of notations, we omit the superscript (j) wherever it is used 
indicate the serial number of the chosen subsystem. 

Define f, = H^i By expanding H^^i (z) as (^) = 1^6^=0 ^s^"* "^i*^ ^0 - 1' 



we have fi = J2s=o ^sVi-s and 



J 



/r = n (2) [Vk, ■ ■ ■ Vk,+i-p 91 (ufcj 



s=l 



A (Z) "0 



1=1 



A (z) 



uLiAsiz)^ 



^s=l jz) /'^^ \^ fn, \ I '■'■s=i y-' c 

A„<o (z) ^' + {z) ^''^'^^ 

p—l '• — -L p— 1 

J J 



s=l s=l 
J J 



s=l 



s = l 



(20) 



where for each d £ {0, • • • , p — 1}, by n'^^' we denote the serial number of the Hammerstein subsystem 
that generates yti-d- Clearly, n^'' € {1, • • • , J}. 
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Using the Cauchy-Schwarz inequality, we see that 



Jp 



<{l + Jp) (E'^') 



(21) 



Thus, in order to prove (111, we need only to show that 



liminf jAmin ( fifl | > a.s. 

\i=0 / 



(22) 



We use the method of reduction to absurdity. If (22 1 were not true, then there would exist a 
measurable set D such that P {D} > and 



liminf hn^in (y M] - 0, Vc^ e D. 



(23) 



We arbitrarily choose ojo E D and fix it. By (23 1 we know that there exist a subsequence {in}„>Q 
of {i}t>o s-J^d a sequence of vectors {'7t„}„>o with \\rit^\\ = 1 such that on the sample path ujq we 
have 



hm y{^lf,Y^O. 



(24) 



Write r]t^ as 



(9.^)1 



(25) 



The boundedness of {?7t„} implies the existence of its convergent subsequence. We arbitrarily choose 
such a subsequence and still use the same notation as {?7t„} to denote it; accordingly, we are able 
to write 



rj = [a> 



^(1.1) 



/3 



(l.r-) 



(9,1) 



(26) 



where ||r/|| = 1. 



From (20 1 and (25) we obtain 
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(0) 



a; 



Au) (z) < 



B u) {z) zcl ° ^ ■■■ a 

"■n 



(0) 



B (i) (2) 2:c) 



A (i) [z) %-i 



p— 1 



+ 



M 



which can be rewritten as 

M M 

T r A 

Vtji = 
where M = Jp + q — 1, 

M 



.m=0 



TO=0 



TO=0 



[51 {Uki) ■■■ 9r {Uki) ^kiV, 



m=0 



(i,™)(i)_m 



m=0 

9-1 



and 



M 



m=0 



m— 



m— 



*" Au>{z} 



(27) 



(28) 



(29) 



(30) 



Recalling the concept intrinsic switch introduced in Section 3, we see that there exist K subse- 
quences {i^"', s >0},K = 1,--- ,K,of {i}~o such that s > 0} fl {i''^^', s > 0} = 0, VI < ki ^ 

K2 < -f'', Uk;=i > 0} = {i}^o ' ^'^'l '^o ■ ■ ■ '^B-i ) ^'^ € {1, • • • , -ftT} , is independent of 



12 



s. Since for each d e {0, • • • ,p — 1} 



, /t^ depends only on k, let us rewrite it as nj^"' from now on. 
Obviously, there exists at least one k E {!,■■■ , K} such that {ig"', s > 0} is an infinite subsequence 
of {«}i^o- Without loss of generality, we may assume that for each k G {1, • • • , K}, s > 0} is 
an infinite subsequence of {iji^o- 
Let us rewrite ( 28 1 as 



M 



r<-)(e>). 



M 



M 



~<o,™,(e)). 



, K^l,--- ,K, 



or, equivalently. 



M 



M 



M 



E '^^r 



m=0 
-1 r 



, /t= I,-- - ,K, 



by noticing that h^^ ^ ° , VZ € {0, 1, • • • , r}, is independent of s. 



Corresponding to ( 29 1 and ( 30 1 , we have 

M 



)(») „rn _ ST^ „ (m) Y[s=l (^) 



m=0 



m=0 
9-1 



E c^''" nil ^ = 1, • • • , r-, K = 1, . . . 



(31) 



(32) 



and 



M 



E f^'C 



m=0 



p-1 

E"* 

rn— 



z™, K = I,-- - 



(33) 



where /i^';'"""' G M, V/ G {0, 1,-- - ,r}, to G {O,-- - ,M}, k G {I,-- - It is seen that 

" ; g 1^ . . . ^ ^} ^ m e {0, . . . , M} , K G {1, • • • , K}} is bounded. 
We now derive from ( [24| that there exist a k G {1, • • • ,K}, an infinite subsequence of {i}t^o' 
and an infinite subsequence of {tn}^Q, where the latter two are denoted by {i^^'j^g and {in}^g, 
respectively, such that 



ii.- i^E 0- 



(34) 



s=0 



Actually, it is obvious that there exist K infinite subsequences of {t}'^Q, denoted by {iJi^'j^g 
and {t'^'}'^^Q, respectively, such that for each n G N, it holds that 



7 ; 



13 



i=0 



and 



K=l S=0 



if 



(35) 



From ( 35 1 and ( 24 1 we see that 



K 4"' 



■ =0 2^K=1 + K=l S = 



^ 0. 



t„ + 

We now show (34 1. Assume the converse: For every k e {1, • • • ,K}, 
Then there exist a positive constant cq and a sufficiently large positive integer N such that 

2 

E(C%)) >coC, V«e{l,--- Vn>iV, 



(36) 



(37) 



(38) 



which leads to 



or 



hmmf — — E (vt / > Co > 0, 



contradicting (36 1. Thus, ( |37[ ) is not true, and we have shown that there exists a k G {1, ■ • • ,K} 
such that 



s=0 



which implies (34 1. 



From now on, let the k in (34 1 be fixed. For simplicity of notations, we omit the superscript 
'" in (34 1 and thereafter: 



(39) 



5=0 
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Recall Lemmas 1 and 2. Arguing similarly to the proof of Theorem 3 in [7 , which is motivated 
by the proof of Theorem 6.2 in [5], we derive from (|33|, and (p9]) that 



/3' 



= 0, 



(40) 



which contradicts ||77|| = 1. Thus 1) is established. 
We now prove 2). 

To this end, recalling (H4), without loss of generality, for each n > 0, we may assume w„ is 
^„-measurable, and therefore by Lemma 4 and 1) of Lemma 5, we need only to show there exists 
a. g > such that 



A, 



(t) = 0(te) a.s. 



(41) 



In fact, by (H5) it follows that 



Amax (t) <tr ^ 



=0 tr^(^,;^[ =oij2 



i=0 
t p-1 



\i=Q 

t g-1 r 



\i=Om=0 1=0 m=0 ; = 1 

=0(0(t'') + 0(t)) = O(i^) U.S., 



(42) 



□ 



where g = max (7, 1); hence, 2) is true and the proof of Lemma 5 is completed. 

Remark 6 It is observed that throughout the proof of Lemma 5, the procedure of deriving (31)- 
(34), which can be characterized as "subsequence partitioning and seeking", plays an important role; 
combining this procedure with the existing techniques applied in the proofs of Theorem 6.2 in |^ 
and Theorem 3 in [Tij leads to the desired result. 
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